Skip to content

Freeze released bundle, control-plan and apply-crash workflows (#271) - #290

Merged
memtomem merged 3 commits into
mainfrom
test/frozen-workflows2b-271
Nov 10, 2026
Merged

memtomem merged 3 commits into
mainfrom
test/frozen-workflows2b-271

Conversation

@memtomem

Copy link
Copy Markdown
Owner

Refs #271. This is the Workflows family, part 2b: the three cases that part 2a (#289) left as "not yet", because each needs new release-written bytes. The issue stays open.

What

  • Generator. freeze.py --part 2b adds scenarios to an existing release directory without writing anything already there. Each new scenario directory and bindings-2b.json must not exist. It runs in a fresh box of its own; part 1's behaviour is unchanged. For both v0.2.0 and v0.3.0:
    • bundle-saved: the worker exits right after the final node's bundle is saved, before the manifest records the node's outcome.
    • bundle-saved-recovered: the same, after the release's own status. In these two the fake kimi reports token usage (opt-in), so the bundle carries some.
    • control-planned, control-completed: Toolgraph control plans on, with a fake toolgraph that answers both protocols from the request it gets (preflight.py is identical in both releases and HEAD).
    • apply-unreceipted, apply-receipted: a completed workflow's final artifact applied in the release interpreter, dying right after the applying record or right before the applied one. This is the artifacts family's crash script.
  • Rebinding. Each scenario is read with its own bindings document. The apply receipt's repository root is rebound (once; it holds no fingerprint).
  • Bundle reclaim on resume:
    • no provider call, and calls_used unchanged;
    • the bundle's usage is added once;
    • the final result's output and usage are rebuilt from the bundle, and the artifact's patch is the released bundle's.
    • A bundle whose patch, commit record or patch digest disagrees is re-run: one more call, and its usage is never counted.
  • Control plans:
    • The control preflight's request is the released control-plan.json bytes, and the evidence is ok with the released digest.
    • Each effective limit is the tighter of current and stored, in both directions. Parallelism is capped by the plan, and node timeouts are the stored ones.
    • A changed candidate list, agent map or gate is refused before any call.
    • With control plans off the plan still bounds the run (one legacy probe for the lead agent); with Toolgraph off, no call at all.
    • A completed one reads as written through show, status, read_control_plan, conversations show (manifest and result) and apply. Its control-preflight.json is left as it is, because no reader revalidates it.
  • Interrupted apply: recover classifies it as completed, not applied, or manual recovery, without touching the approval or the workflow row.
  • docs/architecture/persisted-contracts.md: the Workflows row names these and drops "partly". Still not covered: the TUI's _read_artifact_result.

No source under src/ changes. No current reader misread a released workflow.

Verification

  • Every existing fixture file is unchanged (diff -r against a copy before generation, and git status shows only additions under the fixture tree).
  • suite-run CLEAN; ruff check . clean.
  • 17 injected regressions each fail an intended assertion, and part 2a's 24 still do.
    • Reclaim: reclaim skipped; reclaimed usage dropped; the rebuilt output or usage dropped; each bundle digest check removed.
    • Control plans: binding and structure checks removed; the request rebuilt; the legacy probe skipped.
    • Limits: the current or stored ceiling ignored; the plan's parallelism cap and stored node timeouts dropped.
    • Apply and sidecars: the receipt's apply-id check inverted; conversations show losing its result.
  • One more mutation, skipping node_bundle_is_intact on reclaim, is equivalent at HEAD: read_node_result already verifies the bundle through the same read_node_bundle.
  • One test bug was caught while measuring. A shared parametrize dict was mutated by .pop(), so v0.3.0's gate case silently passed; it was fixed before the first commit.
  • Codex reviewed the design in two rounds. The implementation went three rounds:
    • Round 1: NEEDS-FIX with 3 Major, fixed in 381fbc1. Current limits were never wider than the released ones; the rebuilt final result and the conversations show result were unchecked.
    • Round 2: NEEDS-FIX with 1 Major, fixed in 3e3ed47. The tighter-config case never lowered the output ceiling.
    • Round 3: unprimed, given the diff only. SHIP with no findings.

🤖 Generated with Claude Code

memtomem and others added 3 commits November 10, 2026 07:01
The Workflows family of the persisted-contracts D rows, part 2b: the three
cases part 2a left as "not yet" because they need new release-written
bytes.

freeze.py --part 2b adds scenarios to an existing release directory
without writing anything already there (each new scenario directory and
bindings-2b.json must not exist), from a fresh box of its own. For both
v0.2.0 and v0.3.0:
- bundle-saved: the worker exits right after the final node's bundle is
  saved, before the manifest records the node's outcome; then
  bundle-saved-recovered, after the release's own status. The fake kimi
  reports token usage here (opt-in), so the bundle carries some.
- control-planned, control-completed: Toolgraph control plans on, with a
  fake toolgraph that answers both protocols from the request it gets.
- apply-unreceipted, apply-receipted: a completed workflow's final
  artifact applied in the release interpreter, dying right after the
  applying record or right before the applied one (the artifacts
  family's crash script).

The test reads each scenario with its own bindings document and rebinds
the apply receipt's repository root (once; it holds no fingerprint).
- resume reclaims the released bundle: no provider call, calls_used
  unchanged, the bundle's usage added once, and the final artifact's
  patch is the released bundle's; a bundle whose patch, commit record or
  patch digest disagrees is re-run (one more call, its usage never
  counted).
- run of the released control plan: the control preflight's request is
  the released control-plan.json bytes, its evidence is ok with the
  released digest; effective limits are the tighter of current and
  stored, parallelism capped by the plan, node timeouts the stored ones.
  A changed candidate list, agent map or gate is refused before any call.
  With control plans off the plan still bounds the run (one legacy
  probe for the lead agent); with Toolgraph off, no call at all.
- a released control-completed workflow reads as written through show,
  status, read_control_plan, conversations show and apply; its
  control-preflight.json is left as it is (no reader revalidates it).
- recover classifies a released interrupted apply as completed, not
  applied or manual recovery, without touching the approval or the row.

The doc row names these and drops "partly". Still not covered: the TUI's
_read_artifact_result.

12 injected regressions each fail a test here, and part 2a's 24 still
do. A thirteenth, skipping node_bundle_is_intact on reclaim, is
equivalent at HEAD: read_node_result already verifies the bundle through
the same read_node_bundle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1 found assertions that survived the code they protect:
- every current limit used to be equal to or tighter than the released
  one, so taking the current value outright passed; the released-limits
  cases now run with wider current limits and assert the released
  ceilings, including with control plans or Toolgraph off;
- reclaim checked the totals and the patch but not the final result
  rebuilt from the bundle; its output and its usage
  (TokenUsage.from_mapping) are now captured where the artifact is saved;
- conversations show was checked on its manifest only; its result must
  equal the released result.json.

16 injected regressions now each fail a test here, and part 2a's 24
still do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 2: the tighter-config case never lowered max_output_bytes,
so ignoring the current output ceiling in favour of the stored one
passed. It now runs with 64 KiB and asserts that value.

17 injected regressions now each fail a test here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@memtomem
memtomem merged commit 437c401 into main Nov 10, 2026
11 checks passed
@memtomem
memtomem deleted the test/frozen-workflows2b-271 branch November 10, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant